An Introduction to Plan, Activity, and Intent Recognition

The ability to recognize the plans and goals of other agents enables humans to reason about what other people are doing, why they are doing it, and what they will do next. This fundamental cognitive capability is also critical to interpersonal interactions because human communications presuppose an ability to understand the motivations of the participants and subjects of the discussion. As the complexity of human-machine interactions increases and automated systems become more intelligent, we strive to provide them with comparable intent recognition capabilities. Research addressing this problem is variously referred to as plan recognition, activity recognition, goal recognition, and intent recognition. This synergistic research area combines techniques from user modeling, computer vision, natural language understanding, probabilistic reasoning, and machine learning. Plan recognition algorithms play a crucial role in a wide variety of applications including smart homes, intelligent user interfaces, personal agent assistants, human-robot interaction, and video surveillance. Plan recognition research in computer science dates back at least thirty-five years, initially defined in a paper by Schmidt, Sridharan, and Goodson [64]. In the last ten years, significant new advances have been made on this problem by researchers in artificial intelligence and related areas. These advances have been driven by three primary factors: first, the pressing need for sophisticated and efficient plan recognition systems in a wide variety of applications; second, the development of new algorithmic techniques in probabilistic modeling, machine learning, and optimization (combined with more powerful computers to use these techniques); and finally, our increased ability to gather data about human activities. Recent research in the field is often divided into two subareas. “Activity recognition” focuses on the problem of dealing directly with noisy low-level data gathered by physical sensors such as cameras, wearable sensors, and instrumented user interfaces. The primary task in this space is to discover and extract interesting patterns in noisy sensory data that can be interpreted as meaningful activities. For example, an activity recognition system processing